Iterative and recursive least squares estimation algorithms for moving average systems

An iterative least squares algorithm and a recursive least squares algorithms are developed for estimating the parameters of moving average systems. The key is use the least squares principle and to replace the unmeasurable noise terms in the information vector. The steps and flowcharts of computing the parameter estimates are given. The simulation results validate that the proposed algorithms can work well. 2013 Elsevier B.V. All rights reserved. 1. Problem formulation The least squares methods are effective in modeling physical systems, including the synchronous generator modeling and parameter estimation [1–5]. In general, a system can be modeled by an autoregressive (AR) model, an moving average (MA) model [6], an autoregres sive moving average (ARMA) model [7,8], an impulse response model [9], a Hammerste in nonlinear model [10,11], or a Wiener nonlinear model [12]. The Monte-Carlo simulation tests are used to validate automatic new topic identification of search engine transaction logs [13]. Recently, Ding and Chen proposed a multi-innovation stochastic gradient identification method for linear regression models [14,15] and some related work can be found in [16,17]. The time series models includes three basic models: the autoregressive model, the moving average model and autoregres sive moving average model. Their expansions are the controlled autoregress ive models, the (multiple-input) output error models [18,19], and the multivariabl e ARX-like models [20]. These models play an important role in signal processing [21] and system identification [22,23]. Many parameter identification, adaptive filtering and prediction methods have been reported for the (controlled) autoregressive models and the (controlled) autoregressive moving average models [24–26]. Just as Ding et al. pointed out in [27] that some contributions assume that the moving average processes and the autoregressive moving average processes under consideration are stationary and ergodic and the correlation analysis based methods are not suitable for identifyin g the non-stationary moving average systems and the autoregress ive moving average systems, e.g., [6,8,26]. To this point, Ding, Shi and Chen analyzed the convergence properties of the least squares algorithm and the stochastic gradient algorithm for autoregressive moving average processes [28]. Recently, Wang proposed a least squares based recursive algorithm and a least squares based iterative algorithm for output error moving average systems using the data filtering [29]. This paper studies the identification problems of the non-stationary and non-ergodic moving average systems. Y. Hu / Simulation Modelling Practice and Theory 34 (2013) 12–19 13 This paper is organized as follows. Section 2 derives the identification model of moving average systems. Section 3 presents a recursive least squares algorithm for moving average models. Section 4 derives a least squares based iterative algorithm for identifyin g moving average models. Section 5 provides two examples to show the effectivenes s of the proposed algorithms. Finally, we give some conclusions in Section 6. 2. The identification model Consider the following moving average process: yðtÞ 1⁄4 CðzÞwðtÞ; ð1Þ where y(t) is the system observati on data, w(t) is a stochastic white noise with zero mean, and C(z) is a polynomial in the shift operator z 1 [z w(t) = w(t 1)] with CðzÞ 1⁄4 1 þ c1z 1 þ c2z 2 þ þ cnz : The coefficients ci’s are the parameters to be estimate d from observati on data {y(t): t = 1, 2, 3, . . .}. It is well-known that some identification approaches can estimate the parameters of the moving average processes, e.g., the correlation-ana lysis based algorithms in [8,26] and the recursive least squares algorithms in [28], assuming that w(t) is independen t and stationary and ergodic, and satisfies E[w(t)] = 0, E[w(t)w(t + j)] = 0, j – 0, E[v(t)] = r (constant), where the symbol E represents the expectati on operator. It has been pointed out in [28] that if the variance of the noise w(t) is time-varying, then Eq. (1) is a non-stationar y and non-ergodic moving average process, and the correlation-ana lysis approach es are not suitable for identifying such non-stationary autoregres sive moving average processes. This motivates us to study new identification algorithms for non-stationary moving average processes. This paper proposes an iterative parameter estimation algorithm and a recursive parameter estimation algorithm to identify the parameters ci of the non-stati onary and non-ergod ic moving average processes from available observati on data {y(t)}. Let the superscript T denote the matrix transpose. Define the parameter vector # and the information vector h(t) as